Method and apparatus for copying data from a main site to a remote site

ABSTRACT

A disk subsystem assures the sequence and the coherence of data updating with two or more disk subsystems is provided with an asynchronous remote copy function. A main center  9  includes a computer system having the configuration of slave subsystems  3 - 2  to  3 - n  connected with a master disk subsystem  3 - 1.  It secures the coherence between data at the main center  9  and the remote center  10  at the temporary suspension by repeating temporary suspension and release of the temporary suspension of the remote copy by the master subsystem  3 - 1  at predetermined opportunities. It also repeats temporary suspension and release of the temporary suspension of the remote copy by slave subsystems  3 - 2  to  3 - n  connected to it.

BACKGROUND OF THE INVENTION

[0001] This invention relates to external storage units which store datafor a computer system, as well as to an integrated system of suchexternal storage units, The invention also relates to remote copytechnology to duplicate data among external storage units without needof a host computer connecting external storage unit groups located atdistant places with other external storage unit groups. A group ofstorage units (subsystem) includes a controller that exchangesinformation with a host unit and a storage unit that is integrated witha disk drive for storing data.

[0002] To provide reliable backup of data in a natural disaster such asan earthquake, the distance between a main center and remote centers istypically from 100 km to several hundreds kilometers. For this purpose,several external storage systems provided with a remote copy functionthat duplicate and store the data among subsystems at a main center anda remote center are used. Such remote copy functions can be separatedinto two categories—a synchronous type and an asynchronous type.

[0003] A synchronous remote copy procedure is one that when a dataupdate (write) instruction is issued from a host computer at the maincenter to a subsystem and the object of the instruction is a remote copyfunction, the completion of update processing is reported to the hostunit at the main center after the data update (writing) is completed atthe remote subsystem. In this case, there is a delay time (transmissiontime etc.) corresponding to the geographical distance between the maincenter and the remote center, and the delay can be influenced by theperformance of data transmission lines between the main center and theremote center.

[0004] An asynchronous remote copy operation is a processing procedurethat when a data update instruction is issued from the host unit at themain center to its subsystem and the object of the instruction theremote copy function, the completion of update processing is reported tothe host unit immediately upon completion of update processing of thesubsystem of the main center, with the update (reflection) processing atthe remote center executed asynchronously with the processing at themain center. Therefore, because the data is updated within the timerequired for the processing inside the main center, the processing doesnot require transmission time caused by sending the data to the remotecenter.

[0005] With the asynchronous processes, the contents of the subsystem atthe remote center will not always agree with the contents at the maincenter. Therefore, if the main center loses functionality, for example,due to a natural disaster, the data not already saved at the remotecenter will be lost. The access performance of the subsystems at themain center, however, can be maintained at a similar level to the accessperformance of subsystems which do not execute the remote copy function.Such prior art has had following problems:

[0006] “Maintenance of coherence.” In a remote copy operation, thesubsystem at the remote center is connected with the subsystem at themain center by an independent communication link. When a remote copy isexecuted among the plural subsystems at the main center and the pluralsubsystems of remote centers, the configuration consists of two or moresets of subsystems connected by independent communication links.

[0007] In this configuration, a copy to the remote center is executedfor each communication link; therefore, the time of data update at eachsubsystem is different. If the main center ceases operating, it is notclear which subsystem has data corresponding to the timing of the dataare copied to the remote center. If a time stamp is not attached to eachdata, the coherence between the main center and the remote center willnot be maintained.

[0008] Thus, in executing backup processing at the main center withplural subsystems at the remote center, there is maintenance ofcoherency (maintaining of the updating sequence) of data among theplural subsystems. Here, “sequence” means not only the sequence of data,but also the coherent “state” in the update in data of each subsystem atthe remote side. In an asynchronous remote copy, the reflection ofupdate data to the remote center is delayed from update processing ofdata at the main center. The sequence, however, of the update of thedata must be in accordance with the sequence of the update of the dataat the main center. Or looking at intervals, the sequence of data updateprocessing generated at the main center and the sequence of the dataupdate processing at the remote center must be coherent.

[0009] Generally, a database system comprises a database main body, andvarious kinds of log information and control information, and these aremutually related. In a data update, the log information and controlinformation added to the database main body are updated to maintaincoherence of the system. Therefore, if the sequence of the data updateis scrambled, the coherence of these data that related to the updatesequence will also be scrambled. This can cause the entire database tobe compromised.

[0010] “Intervention of host unit.” In an asynchronous remote copy inthe general environment where the main center and the remote center areprovided with two or more subsystems, generally when a host unitprovides updated data to subsystems information related to the sequenceof data update, for example, a time stamp, is added to the data. Basedon such information, the processing to update the data is executed atthe subsystems at the remote. For example, the remote copy disclosed byU.S. Pat. No. 5,446,871 describes a remote copy function with theintervention of a host system.

[0011] In the art disclosed by U.S. Pat. No. 5,446,871, the issue andtransfer of the information for a data update sequence, and theprocessing to reflect the updated data are realized by cooperation ofthe operating system of the host unit and the disk subsystems on theside of the main center, and the data mover software of the host unitand the disk subsystems on the side of remote center.

SUMMARY OF THE INVENTION

[0012] With a prior art asynchronous type remote copy function thesequence of data update between the main center and the remote centercan be assured. With the prior art, however, special software in thehost unit and the subsystem are required, and the cooperation of each isnecessary. For this, special dedicated software is required anddifficult development, set up, inspection of the software, and review ofsystem design with associated CPU load results. Therefore, considerabletime and expense are required to introduce this function.

[0013] To provide an asynchronous remote copy function using only thefunctionality of the subsystems, the coherency of the sequence of dataupdate must be maintained using only the subsystems. There is a problemthat when the data that require the coherence of the sequence of dataupdate are distributed to two or more subsystems, there is no means formaintaining the coherence of the sequence of data update among two ormore subsystems. Moreover, it is desirable to prevent increased overheadcaused by processing inside subsystems to maintain the sequence of dataupdate.

[0014] The present invention provides an asynchronous type remote copyfunction that assures data coherence using the function of thesubsystems, is easier to introduce and does not significantlydeteriorate performance at the main center, and does not introduce newsoftware to the host unit. The subsystems of a main center and a distantremote center are connected each other. When there are two or moresubsystems, one of the subsystems (hereafter a master subsystem) isconnected with all of other subsystems (hereafter a slave subsystem) atthe main center.

[0015] A master subsystem at the main center temporarily suspends theremote copy and releases the temporary suspension of the remote copy.When a user sets these characteristics in the master subsystem inadvance, the operation can be executed at any time during the executionof remote copy. When the subsystem at the main center receives data forupdate from the host unit, the subsystem starts the storage of the datain the subsystem.

[0016] Then, the master subsystem determines if the remote copy of thesubsystem is temporarily suspended. If not, the data will be the objectof transfer to the remote center. A slave subsystem confirms the stateof the master subsystem by the control bits (explained later) when theslave subsystem receives update data from the host unit. Only when it isconfirmed that the master subsystem is not in temporary suspension ofthe remote copy, is update data transferred to the remote center.

[0017] When the slave subsystem confirms the state of the mastersubsystem and detects that the master subsystem has temporarilysuspended the remote copy, the slave subsystem temporarily suspends theremote copy. In the state of temporary suspension, the data transfer tothe remote center is stopped, regardless of the master subsystem or theslave subsystem. Only the main center executes the data updateprocessing, and the data update information is controlled and storedinside each subsystem. After the master subsystem releases temporarysuspension of the remote copy, both the master subsystem and the slavesubsystem restart the data transfer to the remote center. In this case,the updated data at the main center during temporary suspension of theremote copy is also the object of data transfer. Thus, duplication ofdata between the volumes of each subsystem at the main center and theremote center is executed by the asynchronous type remote copy.

BRIEF DESCRIPTION OF THE DRAWINGS

[0018]FIG. 1 is a diagram that illustrates the configuration of a remotecopy system of a preferred embodiment of the present invention;

[0019]FIG. 2 is a flow chart that illustrates the operation of a remotecopy system;

[0020]FIG. 3 is a flow chart that further illustrates the operation of aremote copy system; and

[0021]FIG. 4 is a diagram that illustrates the configuration of a maindisk subsystem and a slave disk subsystem of a main center.

DESCRIPTION OF THE SPECIFIC EMBODIMENTS

[0022] An example of a preferred embodiment of the present inventionapplied to a computer system is described below. FIG. 1 is an example ofconfiguration of the application of the present invention forduplicating information (data) between an arbitrary pair of centers attwo or more data centers equipped with computer systems.

[0023] Connecting one or more units of disk subsystems 3 on the side ofa main center 9 with one or more units of disk subsystems 7 on the sideof a remote center 10 without the intervention of a host computer 1 and8, a remote copy system that duplicates data between the centers isprovided. An example of a connection of disk subsystems operatingwithout the intervention of host unit, for example, is an SAN (StorageArea Network). FIG. 4 shows an example of the configuration of the disksubsystem 3 at the main center 9.

[0024] In the main center 9 of FIG. 1, a host unit 1, equipped with acentral processing unit (CPU), that executes data processing isconnected with a disk subsystem 3-1 (a master subsystem) and disksubsystems 3-2 . . . 3-n (slave subsystems) through an interface cable 2that provides a transmission path.

[0025] The disk subsystems 3-1, 3-2, . . . 3-n comprise an interfacecontrol part 11 that exchanges data with the host unit 1, a buffer 12that stores data that is referred to or updated by the host unit 1, aremote copy control information storing part 16 that stores informationrelated to the address of updated data during the remote copy istemporarily suspended, a magnetic disk drive 13 that records the data, amicroprocessor 14 that controls these data exchanges, and a disk arraysubsystem control part 17 that controls each of these elements.

[0026] In addition to the components described above, the master disksubsystem 3-1 is provided with a service processor panel 15 for a userto control the remote copy. The remote copy control information storingpart 16 stores control bits that indicate the state of the currentremote copy state from the control information set with the serviceprocessor panel 15. Adding to the information related to the address ofthe updated data during the remote copy is temporarily suspended.

[0027] The master disk subsystem 3-1 is connected with a disk subsystem7-1 of a remote center 10 through an interface cable 4-1. The slave disksubsystem 3-2 is connected with a disk subsystem 7-2 of a remote centerthrough an interface cable 4-2 and the slave disk subsystem 3-n isconnected with a disk subsystem 7-n of the remote center through aninterface cable 4-n similarly. The interface cables 4-1, 4-2, . . . 4-ncan be general communication lines using a network connection unit. Inthis embodiment, these are described as interface cables 4-1, 4-2 . . .4-n.

[0028] When there are two or more disk subsystems 3, the disk subsystem3-1 is connected with a disk subsystems 3-2, . . . 3-n (one other thandisk subsystem 3-1) that stores the data that are the object of theremote copy inside a main center 9 through the interface cable 5. Thus,on the side at the main center 9, the disk subsystem 3 that stores thedata that is the object of the remote copy comprises a master disksubsystem 3-1 and other slave disk subsystems 3-2, . . . 3-n connectedby the interface cable 5.

[0029] When the host unit 1 issues data write request to the master disksubsystem 3-1, the master disk subsystem 3-1 writes the correspondingdata synchronously in the data buffer 12 of its own subsystem andinstructs the data to be written to the disk subsystem 7-1, located at adistant place, asynchronously of the writing of data in the data buffer12. Corresponding data written in the data buffer 12 of its ownsubsystem is recorded in the magnetic disk drive 13 synchronously orasynchronously.

[0030] Because the remote copy method writes data to the distant placeasynchronously, there is a mode that the disk subsystem 3 at the maincenter 9 transmits the update data to the disk subsystem 7 of the remotecenter 10, to which the subsystem is connected, according to thesequence of the update of the volume inside the own subsystem. The disksubsystem 7 in the remote center 10 stores the updated data in thevolume inside own subsystem according to the sequence of the receipt.There is also a mode that the main center 9 transmits the data that isthe object of transfer by a lot at the opportunity optimally scheduledby the disk subsystem 3, not depending on the sequence of the update ofthe volumes in the own subsystem, and the disk subsystem 7 of the remotecenter 10 reflects the updated data to the volumes inside its ownsubsystem regardless of the sequence of the receipt.

[0031] When the host unit 1 issues a data write request to the disksubsystems 3-2, . . . 3-n, the slave disk subsystems 3-2, . . . 3-nwrite the corresponding data synchronously in the data buffers 12 insidetheir own subsystems and then refer to the state of the remote copycontrol information storing part 16 of the master disk subsystem 3-1.The slave disk subsystem judges whether to instruct the data write tothe disk subsystem 7-2, . . . 7-n asynchronously of the writing of datain the data buffer 12 inside its own subsystem or to store theinformation regarding to the storing position of the update data in theremote copy control information storing part 16 inside its own subsystemdepending on the state of the remote copy. The disk subsystems 7-1, 7-2,. . . 7-n are connected with the disk subsystems 3-1, 3-2, . . . 3-nthrough the interface cable 4 and store the data received from the disksubsystems 3-1, 3-2, . . . 3 n in the data buffers 12 inside their ownsubsystems. That is, it shows the system configuration that when hostunit 1 issues write data instruction to one or more of the disksubsystems 3-1, 3-2, . . . 3-n, the same data are stored in one or moreof the disk subsystems 7-1, 7-2, . . . 7-n in the remote center 10depending on the state of the disk subsystem 3-1. Arrows on FIG. 1 showthe flow of the data for writes instructed by the host unit 1.

[0032] Because the master disk subsystem 3-1 has control bits thatindicate the state of the remote copy inside the remote copy controlinformation storing part 16, a system operator can suspend the remotecopy state temporarily by altering the control bits at a predeterminedopportunity or at any time by instruction of the system operator. Whenthe remote copy is temporarily suspended, the disk subsystems 3-1, 3-2,. . . 3-n store the update data in the data buffer 12 of their own disksubsystems, retain the information of the address of the update dataregarding to the write instruction received after the start of thetemporary suspension of the remote copy in the remote copy controlinformation storing part 16, and do not issue, but suspend the writeinstruction of the update data to the disk subsystems 7-1, 7-2, . . .7-n.

[0033] With the present invention, therefore, the data on the side atthe main center 9 at the moment of the temporary suspension of theremote copy reside in all subsystems on the side of the remote center10. That is, the coherence between the data on the side at the maincenter 9 and the data on the side of the remote center 10 at the time oftemporary suspension can be secured. Therefore, the necessity of addingtime stamps to the data for securing coherence is eliminated and theremote copy is realized without the intervention of a host unit, even inan open system where time information is not attached from a host unit.The disk subsystems 3-1, 3-2, . . . 3-n can release the temporarysuspension of the remote copy based on an instruction sent to the masterdisk subsystem 3-1 by the system operator or an instruction by thesystem operator at any time.

[0034] When the temporary suspension of the remote copy is released, thedisk subsystems 3-1, 3-2, . . . 3-n issue the write instruction of thedata that is updated during the temporary suspension to the disksubsystems 7-1, 7-2, . . . 7-n. If the data write request is issued fromthe host unit 1 to the disk subsystems 3-1, 3-2, . . . 3-n, the disksubsystem 3-1, 3-2, . . . 3-n write the corresponding data synchronouslyto the data buffer 12 inside their own subsystem, and further, instructthe data write to the disk subsystems 7-1, 7-2, . . . 0.7-nasynchronously of the data write to the internal data buffer 12.

[0035] With such a configuration, the same data are held within both thevolumes of the disk subsystem 3 that is the object of remote copy insidethe main center 9 and the volumes of the disk subsystem 7 inside theremote center 10 (if the delay of the update timing can be ignored).During the temporary suspension of the remote copy state with the mastersubsystem 3-1, the state of the data of each disk subsystem 3 at themain center 9 at the time the master subsystem 3-1 is temporarilysuspended, that is, the state of the data secured with coherence at thecorresponding time point, are assured and sustained by each disksubsystem 7 of the remote center 10.

[0036] Temporary suspension of remote copy or release of the temporarysuspension of remote copy can be set at the unit of a volume pair. Iftwo or more volume pairs are set as a volume group, changing the statewith the unit of a volume group is enabled. By displaying temporarysuspension or release of temporary suspension on a console of anysubsystems 3 or 7 or the console of host unit 1 or 8, or on a monitorthat is used for the control of these systems, a user can recognizewhether remote copy is currently executed or not and with what unit theremote copy is executed. The user can arbitrarily set the intervalbetween the temporary suspension and the release of the temporarysuspension as long as the interval is not too short for new data copiedto the side of remote center 10 at the release of the temporarysuspension before all of the data before the temporary suspension can becopied to the side of the remote center 10, and the coherence betweenthe data on the side at the main center 9 and on the side of the remotecenter 10 can be maintained.

[0037] As an example of copying time consider the storing of the data ofthe subsystem 7 at the moment of temporary suspension of the remote copyinside the remote center 10, a cycle of the execution of the remote copyfor 30 minutes from the main center 9 to the remote center 10, thetemporary suspension for 30 minutes and then, the execution of theremote copy for 30 minutes after the release of the temporarysuspension. The time of the temporary suspension can be changed inconjunction with the copying time in case the copying time inside theremote center 10 is not 30 minutes, and the interval between thetemporary suspension and the release of the temporary suspension can beset without retaining the copying time.

[0038] There can be a configuration that makes unnecessary the inquiryof the control bits from the slave disk subsystems to the master disksubsystem. This is achieved by distributing control bits that indicatethe remote copy state and are stored in the remote copy controlinformation storing part 16 inside the master disk subsystem 3-1 inadvance from the master disk subsystem to the slave disk subsystems 3-2,. . . 3-n through the interface cable 5. In this case, the state controlof the slave subsystem itself is stored in the remote copy controlinformation storing part 16 of the slave disk subsystem, just as withthe remote copy control information storing part 16 of the master disksubsystem.

[0039] The host unit 8 is a central processing unit that is connectedwith the disk subsystems 7-1, 7-2, . . . 7-n through the interface cable6 at the remote center 10, and refers to and updates the disk subsystems7-1, 7-2, . . . 7-n. The host unit 8 can execute the processingreplacing the host unit 1 when the host unit 1 at the main center 9cannot execute original function by hazard or failure. Further, the hostunit 8 can execute the processing different from those of the host unit1 in the main center 9 independent of the host unit 1 using the datastored in the disk subsystems 7-1, 7-2, . . . 7-n.

[0040] When the host unit 8 does not execute processing to the disksubsystems 7-1, 7-2, . . . 7.n, and when the host unit 8 is not providedwith the replacing function of the host unit 1, the host unit 8 is notnecessary. On the contrary, the main center 9 of FIG. 1 can function asthe remote center and the remote center 10 as the main center, with theconfiguration that is provided with the host unit 8, of which disksubsystem 7-1 is connected with the other disk subsystems 7-2 to 7-nthrough the interface cable 5, as with the similar configuration withthe master disk subsystem 3-1 at the main center 9.

[0041] The method of data duplication and the operation is describedusing FIG. 2. In advance, an operator selects files and volumes, and thedisk subsystem 3 that stores the data that are the object of duplicationaccording to the need of the duplication, that is, the remote copy. Theoperator sets relations among the object files, the object volumes, thedisk, subsystem 3 and the files, the volumes and the disk subsystem 7that are to store the duplication of the selected data, and whether thefull time maintenance of the coherence of the sequence of update isnecessary, to the remote copy control information storing part 16 insidethe master disk subsystem 3-1 from the host unit 1, a service processor15. Generally, in many cases, setting of full time maintenance of thecoherence of the sequence of updates is limited to the log files thatare the history of updates of the database, but in this example, the setup of the full time maintenance of the sequence of update is notexecuted regardless file type.

[0042] In the master subsystem 3-1, the opportunity for temporarysuspension of remote copy and the opportunity of releasing the temporarysuspension of remote copy are set up. As the setting of theopportunities can be executed by the instruction from the host unit 1,the instruction from the host unit 1 can be scheduled in advance by theprogram of the host unit 1.

[0043] For a disk subsystem 3 that can be connected with or equippedwith a dedicated console or the service processor 15 in the selection orsetting as described above, the setting can be done through the consoleor the service processor 15 without the use of the host unit 1. In thisexample, the operator sets the periodical execution of the temporarysuspension, and the release of the temporary suspension, of the remotecopy beforehand at the master subsystem 3-i utilizing the retained timespan inside the disk subsystem 3, without utilizing the host unit 1.

[0044]FIG. 2 is a flow chart that shows the execution of the selectionand the setting from a dedicated console. The temporary suspension andthe release of the temporary suspension of the remote copy are set upwith a volume pair unit that is the object of the remote copy (step 1 inFIG. 2, etc.). Generally, all volume pairs that are the object of theremote copy are defined as a volume group and all volumes inside thevolume group are set to have the same status.

[0045] In this example, all of the volumes of the disk subsystem 3 arethe object of the remote copy. Therefore, the following describes thestate of the remote copy with the unit of the disk subsystem 3, but notwith the unit of a volume pair or a volume group. This detail is notdescribed in this example, but the volume groups are set separately fora database and a log file. Therefore, there can be a definition that theopportunity for temporary suspension or the opportunity for release ofthe temporary suspension of the remote copy is not set up for the volumethat store the log file.

[0046] Because of the method of setting files and volumes that are theobject of the remote copy, there is a method of assigning specificaddresses that implies volumes or disk subsystems, or a method ofselecting an address within the arbitrary range of the addresses by thecontrol program inside the disk subsystem. As the initial setting, theexample of a path setting, a pair setting, and setting the temporarysuspension and release of the temporary suspension are described.

[0047] When the host unit 1 issues a data write request (a writecommand, hereafter) to the disk subsystem 3-1 (step 2), the disksubsystem 3-1 executes data storage processing in the own disk subsystembased on the write command (step 3). After completing data write(storage) processing in the own disk subsystem, the disk subsystem 3-1reports the completion of the write command to the host unit 1 (step 4).

[0048] When the host unit 1 issues write commands to the disk subsystems3-2 . . . 3-n (step 2), the disk subsystems 3-2 . . . 3-n execute thedata storage processing in their own disk subsystems based on the writecommands (step 5). Here, the write command is the command for the datawriting instruction and the transfer of the write data itself, and theuser sets the disk subsystem to which the request is issued in advanceto the host unit 1 (step 1).

[0049] When the disk subsystem 3-1 receives the write command, the disksubsystem 3-1 refers to the control bits inside the remote copy controlinformation storage part 16 that indicate the state of the remote copyof the own subsystem and judges the state of the remote copy of the ownsubsystem (step 6 of FIG. 3). If its own subsystem is in the temporarysuspension of the remote copy, the disk subsystem 3-1 does not transmitthe update data to the disk subsystem 7-1 of the remote center 10, butretains the information regarding the storage position of the updateddata inside its own subsystem (step 7). If its own subsystem is not inthe temporary suspension of the remote copy, the disk subsystem 3-1issues the write command to the disk subsystem 7-1 at the opportunitydecided based on the processing capability of the own subsystem (step8).

[0050] If its own subsystem retains the storage position information ofthe data that are updated during the temporary suspension state of theremote copy, the disk subsystem 3-1 judges the data of the correspondingposition to be the object of transmission to the disk subsystem 7-1 ofthe remote center 10, issues a write command to write correspondingdata, and after completion of processing the write command, erases theupdate position information. When the write command is received, thedisk subsystems 3-2 . . . 3-n issue a command to the disk subsystem 3-1through the interface cable 5 to inquire about the state of the disksubsystem 3-1, and by obtaining and referring to the control bits thatindicate the state of the remote copy of the disk subsystem 3-1 (step9), confirm whether the disk subsystem 3-1 is in temporary suspensionstate of the remote copy (step 10).

[0051] If the disk subsystem 3-1 is in the temporary suspension state ofthe remote copy, the disk subsystems 3-2 . . . 3-n retain theinformation related to the storage position of the updated data insidetheir own subsystems (step 12) and report the completion of theprocessing of the write command to the host unit 1 (step 13). If thedisk subsystem 3-1 is not in the temporary suspension state, the disksubsystems 3-2 . . . 3-n report the completion of the processing of thewrite command to the host unit 1 (step 14) and issue the write commandto the disk subsystems 7-2 . . . 7-n at the opportunity decideddepending on the processing capability of the own subsystems. If thedisk subsystems 3-2 . . . 3-n retain the storage position information ofthe data updated during the temporary suspension state of the remotecopy, the disk subsystems 3-2 . . . 3-n judge the data of thecorresponding position to be the object of transmission to the disksubsystems 7-2 . . . 7-n of the remote center, issue a write command towrite the corresponding data (step 15), and erase the update positioninformation after the processing of the write command is completed.

[0052] That is, when the disk subsystem 3-1 is in the temporarysuspension state of the remote copy, the other disk subsystems at themain center 9 that are connected with the disk subsystem 3-1 turn intothe temporary suspension state of the remote copy due to the issuing ofa write command from the host unit 1. When the disk subsystem 3-1 is notin the temporary suspension state of the remote copy, the other disksubsystems at the main center 9 that are connected with the disksubsystem 3-1 execute the remote copy due to the issuing of a writecommand from the host unit 1.

[0053] When the remote copy state of the disk subsystem 3-1 is changed,the disk subsystems 3-2 . . . 3-n inform the disk subsystem 3-i (step9′, not shown) or as above mentioned, when the remote copy state of thedisk subsystem 3-1 itself is changed, the disk subsystem 3-1 inform thedisk subsystems 3-2 . . . 3-n is possible, instead of the inquiring fromthe disk subsystems 3-2 . . . 3-n to the disk subsystem 3-1 (step 9).

[0054] In case of the above mentioned settings, the disk subsystems 3-2. . . 3-n need to retain the state of the remote copy of themselves aswith the disk subsystem 3-1. That is, when the disk subsystem 3-1 is inthe temporary suspension state of the remote copy at step 10, the disksubsystems 3-2 . . . 3-n change the state of remote copy of their owndisk subsystems to the temporary suspension state of the remote copy(step 11, not shown). When the disk subsystem 3-1 is not in thetemporary suspension state of the remote copy at step 10, the disksubsystems 3-2 . . . 3-n change the state of the remote copy of the owndisk subsystems to the released temporary suspension state of the remotecopy (step 11′, not shown). When the indication of the remote copy stateof the disk subsystems is desired, the disk subsystems 3-2 . . . 3-ninquire to the disk subsystem 3-1 as step 9, the state of the remotecopy can be retained in the own disk subsystems and step 11 and step 11′can be arranged.

[0055] When the disk subsystem 7-1, 7-2, . . . 7-n confirm the receiptof a write command issued from the disk subsystems 3-1, 3-2, . . . 3-n,the disk subsystems 7-1, 7-2, . . . 7-n execute the processing of thewrite command, that is, the data storage processing into the data buffer12 inside the own subsystems (step 16). After completing processing ofwrite command, that is the storage of the data in the data buffer 12 ofits own subsystems, the disk subsystems 7-1, 7-2, . . . 7-n report thecompletion of the processing of write command to the disk subsystems3-1, 3-2, . . . 3-n (step 17).

[0056] When the temporary suspension state is released, the disksubsystems 3-1, 3-2, . . . 3-n issue the write instruction of the dataof the corresponding position based on the storage position informationof the data that are updated after the time the remote copy of its ownsubsystems were temporarily suspended to the disk subsystems 7-1, 7-2, .. . 7-n. When the host unit 1 issues the write request of data to thedisk subsystems 3-1, 3-2, . . . 3-n, the disk subsystems 3-1, 3-2, . . .3-n, write the corresponding data to the data buffers 12 of their ownsubsystems synchronously with the write request, and instruct the datawrite to the disk subsystems 7-1, 7-2, . . . 7-n that are remotelylocated, asynchronously of writing data to the data buffers 12 insideits own subsystems.

[0057] With the present invention, the data written by the host unit 1is not only stored in the disk subsystems 3-1, 3-2, . . . 3-n, but alsocopied and stored in the disk subsystems 7-1, 7-2, . . . 7-n. The stateof the data of the disk subsystems 3-1, 3-2, . . . 3-n at the time pointthe disk subsystem 3-1 is in the temporary suspension state of theremote copy is generated by the disk subsystems 7-1, 7-2, . . . 7-n onthe side of the remote center 10. If data at the main center 9 iscorrupted, the recovery work can be performed utilizing the data in thedisk subsystems 7-1, 7-2, . . . 7-n and the operation can be restarted.

[0058] On the side of the remote center 10, while the remote copy is inthe temporary suspension state, a copy of the data of the disksubsystems 7-1, 7-2, . . . 7-n is made and preserved using the volumecopy function of the disk subsystem. By making the copy, if data at themain center 9 becomes corrupted while executing the remote copy and thecoherence of the data is lost as the data are being written to the disksubsystems 7-1, 7-2, . . . 7-n, the recovery work as the rerun of jobbased on the copy of the preserved corresponding data can be performed.All of these features are realized with the function of the disksubsystems and do not load the processing capability of the host unit.

[0059] As described, a remote copy system that assures the coherence ofupdated data within the range of user expectation and is easy tointroduce without the need of introducing new software and without achange of function on the system side is realized. Furthermore, a highperformance and easy to introduce emergency back up system can beprovided by the improvement of the data transfer efficiency and by thereduction of the controlling overhead of the subsystem for keeping thecoherence of the sequence of update as the data transfer between themain center and the remote center.

[0060] The preceding has been a description of the preferred embodimentof the invention. It will be appreciated that deviations andmodifications can be made without departing from the scope of theinvention, which is defined by the appended claims.

What is claimed is:
 1. A system comprising: a master disk subsystemhaving a controller that exchanges data with a host unit, a storagedevice that stores the data, and a control function that transfers andinterrupts the transfer of the data to disk subsystems located remotely,at a predetermined interval; at least two slave disk subsystems providedwith a controller that exchanges data with the host unit, a storagedevice that stores the data, a transmission channel connected to themaster disk subsystem, an inquiry means that determines whether themaster disk subsystem is in data transfer state or in an interrupteddata transfer state and a means that transfers the data to a disksubsystem that is located remotely when the inquiry result determinesthat the master disk subsystem is in data transfer state, and at leasttwo remote disk subsystems having a controller that is located remotelyfrom the master disk subsystem and the slave disk subsystems andreceives data transferred from the each subsystem, and a storage devicethat stores the data; and the system secures the coherence between thedata inside the master subsystem and slave subsystems and the datainside the remote disk subsystems by transferring the data from themaster disk subsystem and slave disk subsystems intermittently to theremote subsystems.
 2. A method of performing a remote copy to store datain a remote center using a slave subsystem comprising: transferring datato the slave subsystem from a host unit; storing the data in a memory ata storage position; determining whether a master subsystem is in a stateof transferring data to a remote center; transferring the data to theremote center when the master subsystem is in the state of transferringdata, and retaining the information indicative of the storage positionof the data when the master subsystem is not in the state oftransferring data; and transferring the data to the remote center afterthe master subsystem is in the state of transferring data.
 3. Asubsystem that is provided with a control part that exchangesinformation with a host unit, a storage part that stores theinformation, and a transfer controller that transfers the information toother subsystems temporarily suspending the transfer at specifiedopportunities.
 4. A subsystem as in claim 3 wherein the subsysteminitiates the temporary suspension and releases the temporary suspensionfrom time to time.
 5. A subsystem as in claim 3 in which the subsystemretains information about a storage position of update information to betransferred inside its own subsystem at the time of temporary suspensionuntil the time of release of temporary suspension.
 6. A first and asecond subsystem each comprising a control part that exchangesinformation with a host unit, a storage part that stores the informationand a transfer part that transfers the information from the host unit toanother subsystem; and a transmission line that connects the subsystemswith a master subsystem.
 7. A system as in claim 6 in which thesubsystems obtain information about a transfer state of the firstsubsystem through the transmission line.
 8. An integrated systemcomprising: at least a first and a second disk subsystem, each of whichis provided with a control part that exchanges data with a host unit anda storage part that stores the data, and wherein each subsystemtransfers the data from the host unit to a remotely located disksubsystem; and the remotely located disk subsystem determines whetherthe first disk subsystem can transfer data to the remotely located disksubsystems.
 9. An integrated system as in claim 8 wherein the remotelylocated disk subsystem determines whether the transfer is to be executedby determining whether the first disk subsystem is temporarilysuspended.
 10. An integrated system as in claim 9 in which the firstdisk subsystem performs the temporary suspension and the release of thetemporary suspension periodically.
 11. An integrated system as in claim8 in which data transfer between the first disk subsystem connected withthe host unit and the remote disk subsystem is performed asynchronously.12. An integrated system comprising a first disk subsystem groupconnected with a host unit and a second disk subsystem group thatreceives a data transfer from the first disk subsystem group, and inwhich one of the disk subsystems of the first disk subsystem group is amaster subsystem that controls the data transfer from the first disksubsystem group to the second disk subsystem group.
 13. An integratedsystem as in claim 12 in which control of the data transfer is based onwhether the master subsystem temporarily suspends the data transfer tothe second disk subsystem group.
 14. An integrated subsystem as in claim12 in which the first disk subsystem group is connected with the seconddisk subsystem group through a storage area network.
 15. An integratedsubsystem as in claim 12 in which the data transfer between the firstdisk subsystem group and the second disk subsystem group is performedasynchronously.
 16. An integrated system as in claim 15 in which needfor request of an update sequence assurance is determined by assigning adedicated disk volume at time of an asynchronous data transfer.
 17. Amethod of preserving coherence of data that secures the coherence ofdata with a master subsystem by receiving data from the host unit,storing the data in a storage device and inquiring the master subsystemwhether the master subsystem is in the state to transfer data to aremote center.